1. Field of the Invention
The present invention relates to setting of an internal parameter or an external parameter of a camera.
2. Related Background Art
In order to calibrate an internal parameter, such as a focal distance or a principal point, of a camera or an external parameter that is the positional posture of the camera, there is used a method in which a plurality of fiducial points (marks), which are arranged in a three-dimensional space coordinate system and whose coordinate values are already known, are picked up with the camera and an estimation is performed based on a camera calibration algorithm using the space coordinate value and image coordinate value of each mark.
In order to use this method, it is required to associate the space coordinate value of each mark with the image coordinate value thereof. Therefore, there is used a method in which this associating operation is manually performed through human judgment of a mark image. Alternatively, there is used a method in which the associating operation is automatically performed using an image processing technique. In the camera calibration algorithm, there is used a method that is based on marks not all of which are distributed on one plane (meaning that the marks are three-dimensionally distributed) or a method that is based on marks that are two-dimensionally distributed (on a plane). These methods are widely and generally used because of their advantages. For instance, it is possible to arrange marks in a space with ease. Also, it is possible to evolve these methods to a method in which it is also possible to perform calibration of a lens distortion that is a non-linear internal parameter.
This was proposed by Roger Y. Tsai et al. in old times (see Roger Y. Tsai, “A versatile Camera Calibration Technique for High-Accuracy 3D Machine Metrology Using Off-the Shelf TV Cameras and Lenses”, IIIE J. Robotics and Automation, Vol. RA-3, No. 4, pp. 323 to 344, 1987). Aside from this, a large number of proposals are made in Japanese Patent Application Laid-open Nos. 06-137840, 2001-325069, and the like.
As a method of associating the space coordinate value of each mark with the image coordinate value thereof in the case where this camera calibration method is used, there is a method in which the marks are prepared as a pattern having a specific regularity and the pattern is recognized on an image, thereby automatically performing the associating operation. As the pattern to be used, there are proposed a pattern, in which black circles or ellipses are arranged on a white background in a lattice manner, and a pattern constructed from two or more sets of parallel straight lines (for instance, a checkered pattern (Japanese Patent Application Laid-open No. 05-274426 (U.S. Pat. No. 5,444,799), Japanese Patent Application Laid-open No. 07-98214)).
It is possible to use these methods in the case where it is assumed that the whole pattern is picked up or in the case where although only a part of the pattern is picked up, it is not required to specify which area of the whole pattern is picked up.
In the case where the positional posture of a camera is to be estimated, however, it is required to obtain the positional posture by using a specific position of a pattern in a space as a reference. Therefore, there has been formerly pointed out a problem in that in the case where only a partial area of the pattern is picked up or in the case where the visual axis of a camera with respect to the pattern is completely uncertain, it is impossible to specify an area of the pattern in the space to which the partial area of the pattern in a picked-up image corresponds. As a method of solving this problem, there is disclosed in Japanese Patent Application Laid-open No. 06-137840 a method in which some of elements of a pattern are replaced with directional elements, thereby making it possible to specify an area of the pattern (hereinafter referred to as the “Prior Art A”). In this method, some of circular elements constituting the pattern are replaced with directional elements (elliptic elements, for instance) and distinguishing between the pattern elements is performed using the differences in shape and directional property, thereby specifying a picked-up area.
Also, there is a method in which in the case where a pattern including sets of parallel straight lines is used, within each area surrounded by parallel straight lines (each rectangular square constituting a checkered pattern, for instance), a pattern that is different from the checkered pattern is disposed and the patterns are recognized through pattern recognition processing, thereby specifying a partial area (hereinafter referred to as the “Prior Art B”).
Further, there is a method in which a partial area of a pattern is specified by applying different colors to respective pattern elements and combining color recognition processing with pattern spatial feature recognition processing (hereinafter referred to as the “Prior Art C”).
In the case of the prior art A, at the time of judgment through distinguishing of the directional pattern elements from other pattern elements, it is required to perform this judgment using the difference in shape on an image. This leads to a problem in that because the shape of the pattern in a picked-up image changes due to changing of the positional relation between the camera and the pattern, errors tend to occur in the pattern element judgment due to errors in shape recognition. This problem becomes particularly prominent in the case where circles and ellipses are used as in the case of Japanese Patent Application Laid-open No. 06-137840.
In the case of the prior art B, it is required that a different pattern newly provided to specify a partial area of a pattern is captured by a camera with a sufficient size, so that the density of the parallel straight line sets applied to camera calibration is lowered. In order to perform camera calibration, it is more advantageous that marks are widely distributed on an image and as many marks as possible are used, so that the lowering of the density of the parallel straight line sets that function as marks becomes a problem.
In the case of the prior art C, both of the two problems described above are solved. However, the extraction of color areas is generally unstable in comparison with binarizing processing by lightness, which leads to a problem in that there occurs erroneous detection of pattern elements.